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Abstract. Four different ways of obtaining low-density parity-check codes 
from expander graphs are considered. For each case, lower bounds on the 
minimum stopping set size and the minimum pseudocodeword weight of ex- 
pander (LDPC) codes are derived. These bounds are compared with the known 
eigenvalue-based lower bounds on the minimum distance of expander codes. 
Furthermore, Tanner's parity-oriented eigenvalue lower bound on the minimum 
distance is generalized to yield a new lower bound on the minimum pseudocode- 
word weight. These bounds are useful in predicting the performance of LDPC 
codes under graph-based iterative decoding and linear programming decoding. 

1. Introduction 

Expander graphs are of fundamental interest in mathematics and engineering and 
have several applications in computer science, complexity theory, derandomization, 
designing communication networks, and coding theory [HH]- A family of highly 
expanding graphs known as Ramanujan graphs [31 [3] was constructed with excel- 
lent graph properties that surpassed the parameters predicted for random graphs. 
The description of these graphs and their analysis rely on deep results from math- 
ematics using tools from graph theory, number theory, and representation theory 
of groups [5]. Other authors have investigated non-algebraic approaches to de- 
signing expander graphs and one such construction takes an appropriately defined 
product of small component expander graphs to construct a larger expander graph 
[TJinilT]. Moreover, expander graphs have a special appeal from a geometric view- 
point. Isoperimetric problems in geometry have also been described by analogous 
problems in graphs, and a close connection exists between the Cheeger constant. 
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defined for Riemannian surfaces, and the expansion constant in graphs. Expander 
graphs can be viewed as discrete analogues of Riemannian manifolds. 

In this paper, we focus on one prominent application of expander graphs - 
namely, the design of low-density parity-check (LDPC) codes. Low-density parity- 
check codes are a class of codes that can be represented on sparse graphs and have 
been shown to achieve record breaking performances with graph-based message- 
passing decoders. Graphs with good expansion properties are particularly suited 
for the decoder in dispersing messages to all nodes in the graph as quickly as possi- 
ble. Expander codes are families of graph-based codes where the underlying graphs 
are expanders. That is, every element of the family is an expander and gives rise 
to an expander code. The codes are obtained by imposing code-constraints on the 
vertices (and possibly, edges) of the underlying expander graphs [8l[9l[ini. It has 
been observed that graphs with good expansion lead to LDPC codes with minimum 
distance^ growing linearly with the block length. In fact, one method of designing 
asymptotically good linear block codes is from expander graphs [8] . 

The popularity of LDPC codes is that they can be decoded with linear time com- 
plexity using graph-based message-passing decoders, thereby allowing for the use of 
large block length codes in several practical applications. In contrast, maximum- 
likelihood (ML) decoding a generic error-correcting code is known to be NP hard. 
A parameter that dominates the performance of a graph-based message passing 
decoder is the minimum pseudocodeword weight, in contrast to the minimum dis- 
tance for an optimal (or, ML) decoder. The minimum pseudocodeword weight 
of the graph has been found to be a reasonable predictor of the performance of 
a finite-length LDPC code under graph-based message-passing decoding and also 
linear programming decoding [11] [121 HSl [HI [H]. In this paper, we consider four 
different ways of obtaining LDPC codes from expander graphs. For each case, we 
first present the known lower bounds on the minimum distance of expander codes 
based on the expansion properties of the underlying expander graph. We then ex- 
tend the results to lower bound the minimum stopping set size, which is essentially 
the minimum pseudocodeword weight on the binary erasure channel (BEC), and 
finally, we lower bound the minimum pseudocodeword weight on the binary sym- 
metric channel (BSC). We also examine a new parity-oriented lower bound on the 
minimum pseudocodeword weight over the additive white Gaussian noise (AWGN) 
channel, thereby generalizing the result of Tanner '16] for the minimum distance. 

2. Preliminaries 

We introduce some preliminary definitions and notation that we will use in this 
paper. 

Definition 1. A graph G = (AT, Y\E) is (c, d) -regular bipartite if the set of vertices 
in G can be partitioned into two disjoint sets X and Y such that all vertices in 
X have degree c and all vertices in Y have degree d and each edge e e of G is 
incident with one vertex in X and one vertex in i.e., e = (x, y), a; G A, y G 1". 

We will refer to the vertices of degree c as the left vertices, and to vertices of 
degree d as the right vertices. 

The adjacency matrix of a d-regular connected graph has d as its largest eigen- 
value. Informally, a graph is a good expander if the gap between the first and the 

^The minimum distance of a code is a fundamental parameter that determines its error- 
correction capability. 
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Figure 1. A stopping set S = {vo,Vi,V3,V5} in G. 

second largest eigenvalues of the adjacency matrix is as big as possible. More precise 
definitions will be given later in the paper as needed. Note that for a (c, (i)-regular 
bipartite graph, the largest eigenvalue is \/cd. 

Definition 2. A simple LDPC code is defined by a bipartite graph G (also called, 
a Tanner graph) whose left vertices are called variable (or, codebit) nodes and whose 
right vertices are called check (or, constraint) nodes and the set of codewords are all 
binary assignments to the variable nodes such that at each check node, the modulo- 
two sum of the variable node assignments connected to the check node is zero, i.e., 
the parity-check constraint involving the neighboring variable nodes is satisfied. 

Note that equivalently, the LDPC code can be described by a (binary) incidence 
matrix (or, parity-check matrix) wherein the rows of the matrix correspond to the 
constraint nodes of G and the columns correspond to variable nodes and there is 
a one in the matrix at a row-column entry whenever there is an edge between the 
corresponding constraint node and variable node in G. 

The above definition can be generalized by introducing more complex constraints 
instead of simple parity-check constraints at each constraint node, and the resulting 
LDPC code will be called a generalized LDPC code. 

To analyze the performance of graph-based message passing decoding, certain 
combinatorial objects of the LDPC constraint graph have been identified that con- 
trol the performance of the decoder. When transmitting over a binary erasure 
channel (BEC), it has been shown that stopping sets in the Tanner graph control 
the performance of the message-passing decoder. 

Definition 3. [T7] For a simple LDPC code, a stopping set is a subset set S of 
the variable nodes such that every constraint node that is a neighbor of some node 
s G S is connected to S at least twice. 

The size of a stopping set S is equal to the number of elements in S. A stopping 
set is said to be minimal if there is no smaller sized stopping set contained within 
it. The smallest minimal stopping set is called a minimum stopping set, and its size 
is denoted by Smin- Note that a minimum stopping set is not necessarily unique. 
Figure [T] shows a stopping set in the graph. Observe that {v^, vr, v^} and {1)3, v^, vg} 
are two minimum stopping sets of size s„iin = 3, whereas {vq, f 1, D3, ^5} is a minimal 
stopping set of size 4. 

On the BEC, if all of the nodes of a stopping set are erased, then the graph-based 
iterative decoder will not be able to recover the erased symbols associated with the 
nodes of the stopping set [TT]. Therefore, it is advantageous to design LDPC codes 
with large minimum stopping set size Smin- 



Advances in Mathematics of Communications 



Volume 1, No. 3 (2007), 287-307 



290 



Christine A. Kelley and Deepak Sridhara 



For other channels, it has been recently observed that so called pseudocodewords 
dominate the performance of the iterative decoder (In fact, pseudocode- 

words are a generalization of stopping sets for other channels.) We will now intro- 
duce the formal definition of lift-realizable pseudocodewords of an LDPC constraint 
graph G [12 . However, we will also need to introduce the definition of a graph lift. 
A degree £ cover (or, lift) G of G is defined in the following manner: 

Definition 4. A finite degree £ cover oi G = (V, W; E) is a bipartite graph G where 
for each vertex e y U W , there is a cloud Xi — {xi-^jXi^, . . . ,Xi,} of vertices in 
G, with degixi-) = deg{xi) for all 1 < j < £, and for every {xi,Xj) £ E, there are £ 
edges from Xi to Xj in G connected in a 1 — 1 manner. 

Figure [2] shows a base graph G and a degree four cover of G. 

Definition 5. Suppose that c = (ci_i, Ci_2, ■ • ■ j Ci^^, 62,1, . . . , 62,^, . . . ) is a codeword 
in the Tanner graph G representing a degree i cover of G. A pseudocodeword p of 
G is a vector {pi,P2, ■ ■ ■ ,Pn) obtained by reducing a codeword c, of the code in the 
cover graph G, in the following way: 

~ 1 - ^ » ^ s /C11+C12H hCl.i C2.1+C2,2H |-C2,f N 

C = (,Ci4, . . . , Ci,l, C2,l, . . . , C2.1, ■ ■ ■ ) ^ [ 1 , — 2 , ■ ■ ■ ) — 

{pi,P2,.--,Pn)^P, 

where p, = •••+'■'^ 

The vector c on the left hand side of Figure [2] corresponds to a codeword in the 
degree four cover that is also a codeword in the base graph G, whereas the vector 
on the right hand side corresponds to a codeword in the degree four cover that does 
not correspond to a codeword in the base graph. 

From the above definition, it is easy to show that for a simple LDPC constraint 
graph G, a pseudocodeword p = iPi,P2, ■ ■ ■ iPn) is a vector that satisfies the follow- 
ing set of inequalities: 

(1) 0<Pi<l, for i = 1,2, . . . ,71. 

and, if variable nodes ii, 12, ■ ■ ■ , id participate in a check node of degree d, then 
the pseudocodeword components satisfy 

(2) P^, < P^^' for 1,2,.., d 

k=l,2,..d,k^j 

Extending the above for generalized LDPC codes, it can similarly be shown that 
on a generalized LDPC constraint graph G, a pseudocodeword p = {pi,p2, ■ ■ ■ ,Pn) 
is a vector that satisfies the following set of inequalities: 

(3) 0<pi<l, for i = 1,2, . . . ,71. 

and, if variable nodes ii, 12, id participate in a constraint node of degree d 
and that constraint node represents a subcode [d, rd, ed] , then the pseudocodeword 
components satisfy 

(4) (de - l)p,^ < ^'h- , for J = 1, 2, .., d. 

k=l,2,..d,k=ij 

Remark 1. Note that Equation 0] implies that the pseudocodeword components of 
the generalized LDPC constraint graph G also satisfy the following set of inequalities 
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G G 

(base graph) (a degree 4 cover) 





o 

c = (1,1,1,1, 1,1,1,1,0,0,0,0) c = (1,1,0,0, 1,0,0,0,0,1,0,0) 

p = (1,1,0) p = (1/2, 1/4, 1/4) 

Figure 2. A pseudocodeword in the base graph (or a vahd code- 
word in a lift). 

at a degree d constraint node representing a [d, rd, ed] subcode 

(5) p^^ < Pi^, and 
any L^Jj's remaining terms 

(6) K p^. 



any l^jj's remaining terms 

The set of hft-reahzable pseudocodewords can also be described elegantly by 
means of a polytope, called the fundamental polytope [H]. In particular, lift- 
realizable pseudocodewords are dense in the fundamental polytope. For simple 
LDPC codes, equations (1) and (2) are necessary and sufficient conditions for a 
pseudocodeword to lie in the fundamental polytope. However, for generalized LDPC 
codes, equations (|3]), (U, (O, and ([6]) are necessary but, in general, not sufficient 
conditions for a pseudocodeword to lie in the fundamental polytope. 

It was shown in [lllll2j that a stopping set in a simple LDPC constraint graph is 
the support of a pseudocodeword as defined above. Thus, generalizing the definition 
of stopping sets to generalized LDPC code, we have: 

Definition 6. A stopping set in a generalized LDPC constraint graph G is the 
support of a pseudocodeword p of G. 

Note that this definition of stopping sets for a generalized LDPC code implies 
the same operational meaning as stopping sets for simple LDPC codes, i.e., the 
iterative decoder gets stuck if a generalized LDPC code is used for transmission 
over a BEC and the set of erased positions at the receiver contains a stopping set 
(as a subset) as defined above. 

In Sections 3, 4, 5 and 6, we will consider pseudocodewords and their behavior 
on the binary symmetric channel (BSC), and in Section 7, we will consider pseu- 
docodewords on the additive white Gaussian noise (AWGN) channel. The weight 
of a pseudocodeword p on the BSC is defined as follows [18] . 

Definition 7. Let e be the smallest number such that the sum of the e largest 
components of p is at least the sum of the remaining components of p. Then, the 
BSC pseudocodeword weight of p is 

/ \ I -^^i largest ~ X^remaining Pi 

wbsckP) = i - ^ r \^ 

^ ze i, II largest ^ Z^remaining Pi 
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C 



d 



|X| < an 





Inxii > 5c| x| 



ffl parity-check constraint 
([ci,d-l, 2] constraint ) 



n 



Degree c vertices: variable nodes, degree d vertices: simple parity-check constraints. 



Figure 3. Expander code: Case A. 



Definition 8. The minimum BSC pseudocodeword weight of an LDPC constraint 
graph G on the BSC is the minimum weight among aU pseudocodewords obtainable 
from all finite-degree lifts of G. This parameter is denoted by wj^f^^ . 



Definition 9. Let < a < 1 and < (5 < 1. A (c, d)-regular bipartite graph G 
with n degree c nodes on the left and m degree d nodes on the right is an {an, Sc) 
expander if for every subset U of degree c nodes such that |f| < an, the size of the 
set of neighbors of U, |r(t/)| is at least (5c|/7|. 

Let a (c, c?)-regular bipartite graph G with n left vertices and m right vertices be 
an {an, Sc) expander. An LDPC code is obtained from G by interpreting the degree 
c vertices in G as variable nodes and the degree d vertices as simple parity-check 
nodes. (See Figure [3]) 

3.1. Minimum distance. 

Lemma 1. [8] // <5 > 1/2, the LDPC code obtained from the {an,Sc) expander 
graph G as above has minimum distance dmin > o:n. 

3.2. Minimum stopping set size. 

Lemma 2. If 5 > 1/2, the LDPC code obtained from the {an, 6c) expander graph 
G as above has a minimum stopping set size Sjnin ^ an. 

Proof. Suppose the contrary that there exists a stopping set S of size smaller than 
an. Then by the expansion property of the graph, the size of the set of neighbors 
of S is |r(S')| > Sc\S\. The average number of times a vertex in T{S) is connected 
to the set S is < < 2. This means that there is at least one vertex in 
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T{S) that is connected to the set S only once, contradicting the fact that is a 
stopping set. □ 

Note that the above proof is just an extension of the proof of Lemma [T] for the 
lower bound on the minimum distance dmin since it uses the fact that every check 
node neighbor of a stopping set is connected to the set at least twice, which is a 
similar requirement for a codeword in the proof of Lemma [TJ 

3.3. Minimum pseudocodeword weight. 

Theorem 1. If S > 2/3 + 1/Sc such that Sc is an integer, the LDPC code obtained 
from the (an, 5c) expander graph G as above has a pseudocodeword weight 

BSC ^ 2{an-l){36-2) _ 



(26-1) 



Proof. The proof is by contradiction. Let p — {pi, . . . ,p„) be a pseudocodeword 
in G. Without loss of generality, \et pi > p2 > . ■ ■ > Pn- We will show that if e 
is the smallest number such that Pi + P2 + ■ ■ ■ + Pe ^ Pe+i + •■ + Pm then e must 
be more than ^""-yif . We wiU assume a subset U of size e of variable nodes 
corresponding to the e dominant components of p to have a size that is at most 
and establish the necessary contradiction. 
Let V — {vi,V2, ■ ■ ■ ,Vn} be the set of variable nodes. Let U = {vi, . . . ,Ve} 
be a set of e variable nodes corresponding to the e largest components of p. Let 
U = {vi e V\vi i [/, \Y{vi) n r(;7)| > (l - A)c + l}, where T(X) is the set of 
neighbors of the vertices in X and A = 2(1 — 5) + i. Let U' = U LI U. Note that 
since we assume Sc to be an integer, Ac is also an integer. 

We want to show that if \U'\ < an, then we can find a set M of edges such that: 
(i) every node in U is incident with at least Sc edges in M, (ii) every node in U is 
incident with at least Ac edges in M , and (iii) every node in T{U') is incident with 
at most one edge in M. (Such a set M is called a (S, \)-matching in [13].) Suppose 
e = |C/| < -^p^, where /3 = Then by Lemma 6 in [H], \U\ < P\U\. This 

implies that \U'\ < (1 + (3)\U\ < {an — 1). Since G is an (an, (5c)-expander, this 
means |r(i7')| > Sc\U'\ = Sc\U\ + Sc\U\. 

We will prove the {S, A)-matching property in G by constructing a new bipar- 
tite graph G as follows. Label the edges connected to each vertex in [/ U f7 as 
{1,2,..., c}. For each vertex v in U UU, create Sc vertices Vi,V2, ■ ■ ■ , vsc in G. For 
every vertex w in T(U') in the graph G, form a vertex w in the graph G. Let U 
correspond to the set of vertices in G that correspond to the copies of vertices in 
Uuil and let W be the set of vertices in G that correspond to the vertices in T{U') 
in G. For a vertex v in G, connect the vertex Vi € X , i = 1,2, Sc, to a node 
w E W ii and only if the z*'' edge of v is connected to w G T{U') in G. (Note that the 
Sc vertices in G that correspond to a node in U correspond to Sc edges incident on 
that node in G. Furthermore, since G does not contain multiple edges, each of those 
Sc edges are connected to a distinct node in T{U'), which means that each of the Sc 
copies in G corresponding to a node in U are connected to a distinct node in W.) 
Now, for any subset X C U, we will always have that |r(X)| > |X| in G. This can 
be seen by the following argument: since the graph G is an (an, Sc) expander and 
\UUU\ < an, therefore any subset Y C UUU has the property that |r(y)| > Sc{\Y\) 
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in G. Choose Y such that the set of vertices in X in G correspond to the set of 
vertices in Y in the graph G. We have \T{X)\ = |r(y)| > Sc\Y\ > 5c\X\/{5c) = \X\ 
since r(X) — T(Y) by construction and Sc\Y\ > \X\ by construction. Thus, for 
any subset X C U in. the graph G, we have |r(X)| > \X\. Therefore, by Hall's 
(Marriage) Theorem, there is a matching of all nodes in U (which corresponds to 
the Sc copies of vertices in [/ U {/) to the set in W (or r([/')). Since Ac < 6c by the 
choice of A, this means that the matching in G corresponds to a {S, A)-matching for 
the set U' in the graph G. 

Consider all of the check nodes in T{U) that are incident with edges from M 
that are also incident with the vertices in U. Let us call this set of check nodes T. 
We now apply the inequality in equation ^ at each of the check nodes in T and 
combine the inequalities by summing them. For each check node, the left-hand side 
of equation ^ is chosen to be a component of the pseudocodeword corresponding 
to a vertex in U if the edge from M that is incident with the check node is also 
incident with that vertex in U. After combining all such inequalities in all of the 
above check nodes, we obtain an inequality that has Sc{pi + ■ ■ ■ + Pe) on the left 
hand side since there are at least 6c edges from each vertex in U that arc incident 
with M. Furthermore, by the same argument, there are most (1 — 6)c edges from 
each vertex in U that are not in M but are possibly also incident with the above 
check nodes. Moreover, at most (1 — A)c edges from each vertex in U are possibly 
incident with the above check nodes and at most (1 — A)c edges from each vertex 
in V\U' are possibly incident with the above check nodes by the definition of U'. 
Therefore, we have the following inequality when we sum the inequalities obtained 
above at all the above check nodes: 

(7) 5c(pi + ---+pe) < (l-<S)c(pi + ---+pe) + (l- A)c( ^ p,) + (l- A)c( J2 P')- 

The above inequality implies that 

(1 - ^) / 

Pl-\ \-Pe < TT[Pe+l H ^ Pn) < Pe+1 H \-Pn, 

[26- 1) 

from the choice of A. Thus, the desired contradiction is achieved. From the definition 
of pseudocodeword weight on the BSC (Definition [7]), we have w^^'-^{p) > 2e — 1 = 

2{cm-l){3S-2) _ -, 
(25-1) 

□ 



Remark 2. • The proof of the above theorem can also be inferred directly by 
the result in |13j . However, we believe the proof presented here is somewhat 
simpler than the indirect approach in [13] . 
• For the case when 6 = 3/4, the lower bound on the minimum pseudocodeword 
weight Wmin matches the lower bound on d^i^ and Smin presented in Lemmas 
[1] and [21 This is particularly appealing since an expander code achieving the 
lower bound on the minimum distance will also achieve the lower bound on 
the minimum pseudocodeword and will have no pseudocodewords of weight 
less than the minimum distance. 
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|r(x)| > 5c| x| 



ffl generalized constraint 
([d, rd, £d] constraint ) 



n 

Degree c vertices: variable nodes, degree d vertices: sub-code constraints of a [d, rd, ed] code. 



Let a (c, d)-regular bipartite graph G with n left vertices and m right vertices 
be a (an, (5c) expander. (See Definition O) An LDPC code is obtained from G by 
interpreting the degree c vertices in G as variable nodes and the degree d vertices as 
sub-code constraints imposed by a [d, rd, ed\ linear block codeH. A valid assignment 
of values to the variable nodes is one where the (binary) values assigned to the 
variable nodes connected to each constraint node satisfy all the constraints imposed 
by the subcode, meaning that the binary assignments from the variable nodes con- 
nected to each constraint node form a codeword in the subcode. (See Figured) 
Such an LDPC code is called a generalized LDPC code. 

4.1. Minimum distance. 

Lemma 3. [8j //J > \/{ed), the LDPC code obtained from the [an, 5c) expander 
graph G as above has minimum distance dmin > an. 



4.2. Minimum stopping set size. A generalized stopping set is as defined in 
Definition [6] in Section 2. Under the assumption that the [d,rd,ed] subcode has no 
idle components, meaning that there are no components that are zero in all of the 
codewords of the subcode. Definition [6] reduces to the following: A stopping set in 
a generalized LDPC code is a set of variable nodes such that every node that is a 
neighbor of some node s Cz S is connected to S at least ed times. 

Lemma 4. If S > 1/ {ed), the LDPC code obtained from the (an, 5c) expander graph 
G as above has a minimum stopping set size Smin ^ O^Tl. 



^The parameters of an [n, k, d] binary linear block code are the block length n, the dimension 
k, and the minimum distance d. 
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Proof. Suppose the contrary that there exists a stopping set S of size smaller than 
an. Then by Definition [SI there is a pseudocodeword p whose support has a size 
smaller than an. By the expansion property of the graph, the size of the set of 
neighbors of S is |r(S')| > Jcj^j. The average number of times a vertex in T(S) is 
connected to the set S is ^[^L < < de. This means that there is at least one 

1 (6)1 — oc\b\ 

vertex in T{S) that is connected to the set S less than de times. Therefore, there 
are less than de non-zero pseudocodeword components connected to that constraint 
node in T{S). If we choose the de/2 largest components among them, then their 
sum is greater than the sum of the remaining pseudocodeword components at that 
constraint node. This is a contradiction to the inequality in Equation^ meaning p 
cannot be a pseudocodeword and therefore S cannot be a stopping set. Thus, the 
size of S cannot be less than an. □ 

4.3. Minimum pseudocodeword weight. 

Theorem 2. If S > (^^^^^-^ + c{ed+i) ■^^^/i that Sc is an integer, then the LDPC 
code obtained from the [an, Sc) expander graph G has a minimum pseudocodeword 
weight 

BSC ^ 2{an-mde + l)5-2) _ 

^min ^ / -, ^ \ 



{deS - 1) 



Proof. The proof is by contradiction. Suppose G is an (an, (5c)-expander, where 
^ > (de+i) (d£+i)e - Then, assuming p is a pseudocodeword of the LDPC constraint 
graph G, the proof follows that of Case A by choosing a set of variable nodes U 
corresponding to the e dominant components of the pseudocodeword and letting 
\U\ = e< where (3 = (^.^Y)^^^ - We need to show that wgf^ > 2e - 1 = 

2 ian-i)({de+i)S-2) _ gy ^sing a stroug subcode, the S required is less than that 
in Case A, thereby allowing a to be larger and yielding a larger bound overall. The 
argument is the same as in the proof of Case A, where now we set A = 2 — de6 + ^ . 

Following the proof of Theorem 1, we will first show that if = e < -^^^^j 
then \U\ < /3\U\. Suppose to the contrary, \U\ > /3\U\, then that means there 
is some subset t) C U such that \U\ = ll3\U\\ + 1. This means that \U U U\ = 
\U\ + [l3\U\\ +1 < {1 + P)\U\ + 1 = an. Since G is an {an, 6c) expander, we 
then have \T{UUU)\ > Sc{\U\ + \U\). However, observe that \T{UUU)\ = \T{U)\ + 
\r{U)\r{U)\ < c\U\ + {Xc-l)\U\ since \T{U)\ < c\U\ and |r(;7)\r(t/)| < {Xc-l)\U\ 
by definition. Combining the above inequalities, we have (5c(|C/| + \U\) < c\U\ + 
(Ac - l)\U\, implying |;7| < lfsZx)+l = PPl- This contradicts the choice of t) 
above. Thus, if \U\=e< then \U\ < P\U\. 

Following the rest of the proof of Theorem 1 and using the inequality in Equation 
|4] for the pseudocodeword components, the first inequality in equation ([7]) in the 
proof of Case A now becomes 

{de - l)6c{pi + ---+pe) <{l- S)cipi + ---+pe) 

+ (1-A)c(^p,) + (1-A)c( J2 P^)- 
v,£U viev\u' 



Advances in Mathematics of Communications 



Volume 1, No. 3 (2007), 287-307 



Eigenvalue bounds on the pseudocodeword weight of expander codes 



297 



Expander Graph G Generalized LDPC code 




ffl constraint node 

[d,rd,£d] sub-code 
• variable node 



d-regular expander (2, d) -regular bipartite graph 

on N vertices and E edges on E left vertices and N right vertices 

Edges: variable nodes, degree d vcrtiees: constraints of a [d^rd, ed] code. 

Figure 5. Expander code: Case C. 

This yields 

Pi H l-Pe < (^g^ _ ^ ^P"-) < + ■■■+Pn 

from the choice of A. Thus, by Definition [3 the weight of p is w(p) > 2e — 1. □ 

Remark 3. • Since de > 2 for any judicious choice of subcode, the lower bound 
in Theorem 2 is always greater than the lower bound in Theorem 1. Further, 
the graph need not be as good an expander in Case B as in Case A for the lower 
bound to hold. Thus, using strong subcodes is advantageous for constructing 
good LDPC codes from expander graphs. 
• Note that for 6 = , the lower bound on the pseudocodeword weight equals 
the lower bound on the minimum distance and minimum stopping set size. 



5. Case C 

Definition 10. A connected, simple, graph G is said to be a {n,d,iJ,) expander if 
G has n vertices, is d-regular, and the second largest eigenvalue of G (in absolute 
value) is fi. 

Let a c?-regular graph G be an (n, d, fi) expander. An LDPC code is obtained 
from G by interpreting the edges in G as variable nodes and the degree d vertices 
as constraint nodes imposing constraints of an [d, rd, ed] linear block code. (See 
Figure[S]) The resulting LDPC code has block length TV = nd/2 and rate R > 2r— 1. 

We now state a particularly useful result by Alon and Chung [THl [S] describing 
the expansion of a d-regular graph. 
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Lemma 5. (Alon-Chung) Let G he a d-regular graph on n vertices and let ^ be 
the second largest eigenvalue of its adjacency matrix. Then every subset S of jn 
vertices contains at most ^(7^ + ^(7 — 7'^)) edges in the subgraph induced by S in 
G. 

5.1. Minimum distance. 

Lemma 6. |8, The LDPC code obtained from an (n, d, /i) expander graph G as 
above has minimum distance dmin ^ N-k — 

Note that the above result of Sipser and Spielman can be improved by a tighter 
bound in the last step of their proof in [8J to dmin > (\_ m ^ ■ 

5.2. Minimum stopping set size. 

Lemma 7. The LDPC code obtained from an (n, d, /i) expander graph G has a 
minimum stopping set size Smin > iVe ,.. 

Note that we again use Definition [6] for stopping sets in G. 

Proof. Let S" be a subset of variable nodes (edges in G) of size ^(7^ + ^(7 — 7^)) 
representing a stopping set in G. Then 5* is the support of some pseudocodeword 
p in G. By the Alon-Chung lemma, the set S has at least 777. constraint node 
neighbors r(S'). Since each edge in S has two constraint node neighbors in T{S), 
this implies that the average number of edges in S connected to a constraint node 
in T{S) is at most 2^(t'+?(t-t")) ^ However if 

777 d 

then there is at least one node in T{S) that is connected to S fewer than ed times 
to S. That means that fewer than ed non-zero components of p are connected to a 
constraint node. It can now be shown that the inequality in Equation [S] is violated, 
implying that p cannot be a pseudocodeword (and, S is not a stopping set.) 

The above inequality in equation ^ holds for 7 < jzi- Substituting the value 

of 7 in 151 = -^(7^ + d ~ T^))' infer that the graph cannot contain a stopping 

, (e— — ) 

set of size less than ■ Hence, 



2 

□ 

5.3. Minimum pseudocodeword weight. 

Theorem 3. The LDPC code obtained from an (n, d, pL) expander graph G has a 
minimum BSC pseudocodeword weight lower bounded as follows: 

(- - i^) 
yjBSC > ^ [2 dl 
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Proof. For sake of simplicity, we assume that ^ is an integer. However, it is easy 
to extend the proof for any value of de. The d-regular graph G can be transformed 
to a (2, (i)-regular bipartite graph G' by representing every edge in G by a vertex 
in G" {the variable nodes) and every vertex in G by a vertex in G' {the constraint 
nodes) and connecting the variable nodes to the constraint nodes in G' in a natural 
way. The variable nodes have degree two and they represent codebits of the LDPC 
code G, whereas the constraint nodes have degree d and each represents a [d, rd, ed]- 
subcode constraints. 

Let p = {j)i,P2t . ■ tPn) be a pseudocodeword, where = ^ is the number 
of edges in G and also the length of the LDPC code. Without loss of generality, 
let us assume that Pi > P2 ^ • • • ^ Pn- Let e be the smallest number such that 
Pi + P2 ■ • ■ + Pe > Pe+i + • • • +Pn- Let be the set of edges in G that correspond 
to the support of the e largest components of p, and let T{Xe) be the set of vertices 
incident on Xf.. Note that in the transformed graph G', X,. is a subset of the 
variable nodes, and T{Xq) is its set of neighbors. 

Let \Xe\ = ^(7^ + ^(7-7^)), where 7 < (|^). Since G is an (n, d, /i) graph, 
we have |r(Xe)| > ^n. We now claim that there is a set of edges M in G' called 
an e-matching such that (i) every vertex in X^ in the graph G' is incident with at 
least one edge from M and (ii) every vertex in T{Xe) in the graph G' is incident 
with at most (ie/4 edges from M. 

Given the claim, we can apply the pseudocodeword inequality from equation [S] 
at each of the vertices in r(Xe) that is incident with edges from M . For each such 
vertex w in r(Xe), the left-hand side of equation ^ is chosen to have the cZe/4 or 
less components of the pseudocodeword that correspond to the vertices in X^ that 
are connected to w via an edge from M . After combining all such inequalities in all 
of the above constraint nodes, we obtain an inequality that has 3(pi + ■ ■ ■ -\- Pe) on 
the left hand side. 

Furthermore, there is at most one edge from each vertex in that is not in M 
but is possibly also incident with the above constraint nodes in T{Xe). Moreover, 
at most two edges from each vertex mV — Xf, are possibly incident with the above 
constraint nodes. Therefore, after applying the pseudocodeword inequality (equa- 
tion [S]) as above at each of these constraint nodes and summing these inequalities, 
we obtain the following inequality: 

iex^ iex^ i^x^ 

Simplifying, we get 

iex^ it^x^ 

By the definition of the pseudocodeword weight on the BSC channel (see Def- 
inition [T]), we have that the pseudocodeword weight of p is 'w{p) > 2\Xf.\. Since 

l^el = f (7' + ^(7 - 7')), for 7 < (t|), we have w{p) > 2(f )(§ )(|^) = 
- — — . 

Ne{ )■ This proves the desired lower bound on Wmin- 

To prove the claim, observe that for any set X of left vertices in G' such that 
\X\ = iV(7^ + ^(7-7^)) where 7 < (i^l), we have \T{X)\ > 771 > In other 

words, for every X such that l-'^l = ^(7^ + ^(7 ~ 7^)) where 7 < ( jj ), we have 
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|r(X)| > \X\. Thus, for any X C X^, we have 



(9) 



{-mx)\ > \x\. 



We now prove the claim that there is an e-matching for the set Xg using contra- 
diction. The proof is very similar to the converse of Hall's marriage theorem. We 
want to show that there is a set of edges M in G' such that: (i) every v e X^ is 
incident with at least one edge from M and (ii) every w G r(Xe) is incident with at 
most (ie/4 edges from M. We will prove this by showing that there is a matching 
M such that: (i) every v G X^ is incident with exactly one edge from M and (ii) 
every w E r(Xe) is incident with either exactly de/A edges from M or zero edges 
from M. 

We consider the induced subgraph G" of Xe in G' . Suppose to the contrary no 
such matching exists, then we will assume that there is a maximum matching M' 
such that the maximum number of vertices in Xe are matched to the vertices in 
r(Xe) as described above. That is, M' is the maximum matching such that as many 
vertices in Xg are each incident with one edge from AI' and all the vertices in r{Xe) 
are incident with either zero or exactly de/A edges from M'. Since we assume that 
not all the vertices in X^ are incident with edges in M', there is a vertex v G X^ 
that is not incident with any edge from M'. Now, we let S be the set of vertices in 
Xe that are connected to v by an M'-alternating patli in G" and let T be the set of 
vertices in r{Xe) that are connected to v by an M'-alternating path in G" . Then, 
it is clear that S C Xe and T{S) = T. Furthermore, every vertex in S — v has one 
edge incident from M' that is connected to some vertex in T and every vertex in T 
has (ie/4 edges incident from M' that are connected to vertices m S — v. (Since M' 
is a maximum- matching, it is easy to show that there is no M'-augmenting path as 
defined in 231-) This means that (f = \S\ — 1, which contradicts equationlHl 
This proves that there exists a matching M as described above. □ 

The above proof holds even when de/A is not an integer. In that case we simply 
replace f with [fj in the above when de/A > 1. In the case when de/A < 1, the 
proof is trivial since the e-matching condition follows directly from Hall's marriage 
theorem. 

Remark 4. • Note that the lower bound on the minimum pseudocodeword 
weight closely resembles the lower bound on the minimum distance and the 
minimum stopping set size. The only difference is a factor of two in the e term 
within the braces in Lemma [7] and Theorem [3l 
• The lower bound suggests that if one were to use good expanding graphs 
such as the Ramanujan graphs from the construction in [3] and choose an 
appropriate choice of subcodes having minimum distance at least twice the 
second eigenvalue of the expander then the resulting code will have a good 
pseudocodeword weight and a good minimum distance. This is interesting for 
designing codes that are good for iterative decoding or LP decoding. 



■^Refer to |24j for the definition of an A/'-alternating path. 
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C 




ffl [c,r^c,8p] constraint 



K [d,r2d,8^] constraint 



N = mc = nd 



E 



n 



m 



Edges: variable nodes, degree c vertices; [c, ric, eic] constraints, degree d vertices: [d^ r2d^ e2d] constraints. 



Definition 11. A (c, d)-regular bipartite graph G on m left vertices and n right 
vertices is a (c, d, m, n, fj.) expander if the second largest eigenvalue of G (in absolute 
value) is /i. 

Let a (c, (i)-regular bipartite graph G be an (c, d, to, n, /i) expander. An LDPC 
code is obtained from G by interpreting the edges in G as variable nodes, the degree 
c left vertices as sub-code constraints imposed by an [c, ric, eic] linear block code, 
and the degree d vertices as constraint nodes imposing constraints of an [d, r2d, £2^] 
linear block code. (See Figure [6l) The resulting LDPC code has block length 
N = mc = nd and rate i? > ri + r2 — 1. 

We state a useful result by Janwa and Lai 'lU\ describing the edge-expansion of 
a regular bipartite graph G. 

Lemma 8. (Janwa-Lal, edge- expansion) Let G he a {c, d)-regular bipartite graph 
on m vertices on the left and n vertices on the right and let be its second largest 
eigenvalue. If S and T are two subsets of the left and the right vertices, respectively, 
of G, then the number of edges in the induced sub-graph of S and T in G is at most 



6.1. Minimum distance. 

Lemma 9. [10] If e2d > eic > fj,/2, the LDPC code obtained from the {c,d,m,n, fj,) 
expander graph G as above has minimum distance 



Figure 6. Expander code: Case D. 



6. Case D 



\E{S,T)\<-\S\\T\ + i^i\S\ + \T\). 
m 2 
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6.2. Minimum stopping set size. We again use the generalized definition of stop- 
ping set in Definition [51 Under the assumption that the [d,r2d,e2d] and [c, ric, eic] 
subcodes have no idle components, meaning that there are no components that are 
zero in all of the codewords of either of the subcodes, Definition [6] reduces to the 
following: A stopping set in a generalized LDPC code as in Case D is a set of vari- 
able nodes such that every node that is a degree c neighbor of some node s €z S is 
connected to S at least eic times and every node that is a degree d neighbor of some 
node s G S is connected to S at least e2d times. 

Lemma 10. The LDPC code obtained from the (c, d, m, n, fi) expander graph G has 
a minimum stopping set size 

Smin > N £162 ^=(£11/3 + e2\ -) ■ 

\ 2Vcd yd y c'J 

Note that when min{e2rf, eic} > /i, the lower bound in the above is positive and 
meaningful. 

Proof. Let X be a stopping set corresponding to a subset of edges in G and let S 
and T be the set of left and right neighbors, respectively, of X in G. Then X is the 
support of some pseudocodeword p in G. Suppose there is some node in S that is 
connected fewer that cci times to the edges in X, then the inequality in Equation[5] 
is violated by the pseudocodeword components at that constraint node. Similarly, 
if some node in T is connected fewer than de2 times to the edges in X, then the 
corresponding pseudocodeword components will not satisfy all the inequalities in 
Equation O Thus, every node in S is connected to X at least cei times and every 
node in T is connected to X at least de2 times. This means \S\ < ^ and \T\ < 
By Lemma m we have 

|^|<imT)|<-|5||r| + ^(|5| + |T|). 
m 2 

This can be further bounded as 

\X\ < -\S\\T\ + + |T|) < + ^(^ + 

m 2 TO cdeie2 2 cei ae2 

Simplifying, we obtain 

□ 

6.3. Minimum pseudocodeword weight. 

Theorem 4. If e2d > eic, the LDPC code obtained from the {c, d,m,n, ^) expander 
graph G has a minimum pseudocodeword weight 

yjBSC > ^£ (!i „ ^) 



Note that the above lower bound is positive and meaningful when eic > 2/i. 
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Proof. Let p = {pi,p2, ■ ■ ■ ,pn) be a pseudocodeword. Without loss of generahty, 
let us assume that pi > P2 > • • ■ ^ Pw- Let e be the smallest number such that 

(10) P1+V2 hPe>Pe+lH hpAT. 

Let Xf. be the set of edges in G that correspond to the support of the e largest 
components of p. Now we define a set S as the set of left neighbors (degree c 
neighbors) to the edges in Xg, and similarly define a set T as the set of right 
neighbors (degree d neighbors) to X^. The (c, (i)-regular graph G can be transformed 
to a graph G' by representing every edge in G by a vertex (called a left-vertex) 
in G", every vertex of degree c in G by a vertex (called a right-left vertex) in 
G', every vertex of degree d in G by a vertex (called a right-right vertex) in G' 
and by connecting the edges from the left vertices to the right-left and right-right 
vertices in G' in a natural way. The left vertices have degree two and they represent 
variable nodes of the LDPC code G, whereas the right-left vertices have degree c and 
represent [c, ric, eic]-subcode constraints and the right-right vertices have degree d 
and represent [d, r2C?, e2rf]-subcode constraints. Note that T{Xe) = S* U T in G'. 

Let \Xf,\ < — ^). Now let us consider two cases. 

Case 1: Suppose \S[JT\ = \S\ + |T| < Then, Since G is a {c,d,m,n, fx) 

graph, we have 



\X4<^\S\\T\ + (^{\S\ + \T\ 



i\S\ + \T\)^ 

d 16|Xe|2 , flA\Xe 



Note that IS'IITI < i^m£iL. Hence, we have 



XJ < 



m Ac^ef 2 cei 
On simplifying, the above yields 

This inequality contradicts the assumption on the size of X,,- 

Case 2: Suppose \SUT\> ^\Xe\. Then we claim that there is a set of edges 
M in G' called an ei-matching such that (i) every vertex in Xg in the graph G' is 
incident with at least one edge from M and (ii) every vertex in S* U T in the graph 
G' is incident with at most cei/4 edges from M. 

The rest of the proof is similar to that for Theorem 3. Given the claim, it is easy 
to show that by applying the pseudocodeword inequality from equation [6] at all the 
nodes in 5 U T that are incident with edges from M and summing them, we can 
arrive at at an inequality of the form 



This will prove that the weight of p is u'(p) > 2|Xe| implying that the minimum 
pseudocodeword weight is 

<f„^ > 2|Xe| 

H6I1C6 

<.^^>2A^^.,(|-^)=iV^.r(^-^). 

2a 2 c a 2 c 

The proof for the matching also follows the same arguments as that in Theorem 3. 
We derive the condition for proving the matching as follows: For any subset X C Xe, 
let Sx be the set of right-left neighbors of X in G' and let Tx be the set of right- 
right neighbors of X in G'. Note that IS'xUT^I = \Sx\ + \Tx\ > Otherwise, 
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using the argument in case 1 and the fact that G is a (c, d, m, n, /i) expander, it can 
be shown that \X\ > — 7) > l-'^el, which is a contradiction. Thus, for any 

subset X C Xe, we have 

(11) \SxUTx\>—\X\. 

cei 

The rest of the proof is similar to that in Theorem 3. □ 

Remark 5. • Note that if cei > de2 > 2/j,, then it can be shown using a similar 
proof as in Theorem 4 that > N^e2{^ - 

• Observe that the lower bound on the minimum pseudocodeword weight is 
slightly weaker compared to the lower bound on the minimum distance and the 
minimum stopping set size, since the proof in Theorem 4 exploits the strength 
of only one the subcodes - namely, the subcode with the smaller distance. We 
however believe that this can be improved to give a much stronger result as 
stated below. 

• Note that in the case where c — d, m ~ n, and ei = £2 = the result in 
Theorem 4 closely resembles the result in Theorem 3 and is almost equal to 
the lower bounds on the minimum distance and the minimum stopping set 
size. 

• The lower bound suggests that if one were to use good expanding graphs such 
as the bipartite Ramanujan graphs from the construction in [3] and choose 
an appropriate choice of subcodes having minimum distance at least twice 
the second eigenvalue of the expander then the resulting code will have a 
good pseudocodeword weight and a good minimum distance. Once again, 
this is interesting for designing codes that are good for iterative decoding or 
LP decoding. Furthermore, with different choices of c and d, there is greater 
flexibility in the designing good codes using the construction in Case D than 
that in Case C. 

We believe that Theorem 4 can be improved to a stronger result as follows: 

Conjecture 1. If e2d > eic > 2/i, the LDPC code obtained from the {c,d,m,n,fi) 
expander graph G has a minimum pseudocodeword weight 

7. A PARITY-ORIENTED LOWER BOUND 

Definition 12. The weight of a pseudocodeword q = (gi, (72, . . • , ?«) of an LDPC 
constraint graph G on the AWGN channel is defined as [TSl [51] 

/V^^ \2 
„.,AWGN, _ \l^i=l1^) 

~ (Er=i 9?) ■ 

The following bound on the minimum pseudocodeword weight on the AWGN 
channel is an adaptation of Tanner's parity-oriented lower bound on the minimum 
distance .16J . Further, this bound complements the bit-oricnted bound obtained by 
Vontobel and Koetter (55] which is also a lower bound on the minimum pseudocode- 
word weight in terms of the eigenvalues of the adjacency matrix of G, obtained using 
a slightly different argument. 
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Theorem 5. Let G be a connected {j,m) -regular bipartite graph representing an 
LDPC code with anrxn parity check matrix H. Then the minimum pseudocodeword 
weight on the AWGN channel is lower bounded as 

...AWGN > n(4j - 112m) 

min — / \ ' 

where ni = jm and /Z2 are the largest and second-largest eigenvalues (in absolute 
value), respectively, of HH^ . 



Proof. Let q = ((ji, . . . , qn) be a pseudocodeword of G, and let p = H<\ be a real- 
valued vector of length r. The first eigenvector of HH^ is ei = (1, 1, . . . , 1)^ /^/r. 
Let Pi be the projection of p onto the ith eigenspace. We will now upper bound 
II H^p 11^. Converting || H^p ||^ into eigenspace representation, we get 

II H^p r= j^l,, II p, f = Ml II Pl f II Pi f 

i=l i=2 



Note that 



< Ml II Pl II +M2(|| P II - II Pl II )• 



"2 ^ 



|'<mj(^g2 

i=l 



The first equality follows from the choice of p and the regularity of the parity check 
matrix H . The second inequality follows by applying the identity (91 + 92 + • • • + 
QtY < i{<li + si + ' ' ' + 9t ) to the terms in the expansion of || p |p. 
The above set of equations yield 



r 

We now lower bound || H^p 11^ follows 



■ 2 ii 2 ''' 



t=i i=i ^=1 t=i 

This bound may be seen by observing that for each t in the outer summation, 
the inner sums over the indices i and £ contribute j qt terms and (m — l)j terms 
involving other q^s. When t is fixed, for each i wherein hu = 1, we have qt and 
(to — 1) other g^'s that contribute to the inner sum. Since qt and the (to — 1) other 
g/c's are involved in the ith constraint node and since q is a pseudocodeword, we 
have gt+sum of (to — 1) other qk's > 2qt. Since there are j values of i wherein 
hit = 1, for a fixed t, the inner sum over i and £ can be lower bounded by 2jqt. 

Thus, II H^p r> YJUi^jit? = 4j^(e;Li g?)- 

Combining the upper and lower bounds, we get 

(4j^ - tl2>l>j)r (ELl (li? _ AWGNr^ 

(Mi-rf - (Er=i9D 

Since nj = rm, we obtain the desired lower bound. □ 
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Remark 6. Note that this lower bound is not as strong as the bit-oriented bound 
in [22] , It equals the bit-oriented bound for the case when m = 2. However, we 
believe that by a different but judicious choice of p in the above proof and by using 
stronger intermediate bounding steps, a much stronger parity-oriented bound can 
be obtained. 

8. Conclusions 

In this paper, the expander-based (i.e., eigenvalue- type) lower bounds on the 
minimum distance of expander codes were extended to lower bound the minimum 
stopping set size and the minimum pseudocodeword weight of these codes. A new 
parity-oriented lower bound in terms of the eigenvalues of the parity-check ma- 
trix was also obtained for the minimum pseudocodeword weight of LDPC codes on 
the AWGN channel. These lower bounds indicate that LDPC codes constructed 
from expander graphs provide a certain guaranteed level of performance and error- 
correction capability with graph-based iterative decoding as well as linear program- 
ming decoding. Further, the results indicate that if the underlying LDPC constraint 
graph is a good expander, then the corresponding expander code has a minimum 
BSC pseudocodeword weight that is linearly growing in the block length. This is in 
general a very hard criterion to ensure in the construction of good error correcting 
codes at large block lengths. It would be interesting to derive upper bounds on 
the distance, stopping set size, and pseudocodeword weight of expander codes to 
examine how tight the derived lower bounds are. 
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